Aminet 1 (Walnut Creek)/Aminet - June 1993 [Walnut Creek].iso/usenet/sources/volume89/aplictns/anncs.1

Lisp/Scheme | 1989-07-13 | 28KB | 1,064 lines

Path: xanth!ames!apple!sun-barr!newstop!sun!swap!page
From: [e-mail] (Bob Page)
Newsgroups: comp.sources.amiga
Subject: v89i168:  anncs - amiga neural net construction set
Message-ID: <[e-mail]>
Date: 13 Jul 89 02:44:07 GMT
Sender: [e-mail]
Lines: 1053
Approved: [e-mail]

Submitted-by: [e-mail] (Jason Leigh)
Posting-number: Volume 89, Issue 168
Archive-name: applications/anncs.1

The ANNCS is a set of functions that allows quick and easy construction of
neural networks.  The system is very simple and as a result much can be
enhanced.  Eventually I may attach a graphical interface that allows you to
visually create neural networks and convert the images to ANNCS Lisp
function calls.  This is by no means a competitor for the Rochester
Connectionist Simulator but hay, what the heck, it's at least running on my
favorite computer!

# This is a shell archive.
# Remove anything above and including the cut line.
# Then run the rest of the file through 'sh'.
# Unpacked files will be owned by you and have default permissions.
#----cut here-----cut here-----cut here-----cut here----#
#!/bin/sh
# shar: SHell ARchive
# Run the following text through 'sh' to create:
#    and.lsp
#    neural.lsp
#    neural.readme
#    xor.lsp
#    xor2.lsp
# This is archive 1 of a 1-part kit.
# This archive created: Wed Jul 12 19:38:16 1989
echo "extracting and.lsp"
sed 's/^X//' << \SHAR_EOF > and.lsp
X;;; @@FILE and.lsp
X;;; Copyright (C) 1989 By Jason Leigh 5/5/89
X
X;;; Redistribute this program at will.  If you decide to use this program
X;;; as part of some commercial package you are developing, by all means
X;;; proceed, all I ask is that you give me a small amount of credit for it.
X
X;;; Creating an AND network using the functions in neural.lsp
X
X;;; The activation function is Relay and a diagram of this
X;;; network can be found in:
X;;; Bernard Widrow, IEEE Computer, March 1988, p.33
X
X(ClearNetwork)
X
X;;; Select the Relay function as the output function.
X(defun OutputFunction (sum threshold)
X    (Relay sum threshold))
X
X(CreateInputLine 'i1)            ;Labelled as i1
X(CreateInputLine 'i2)            ;Labelled as i2
X(CreateNeuron 'n1 0)            ;Labelled as n1
X                    ;This is the output neuron.
X(CreateInputLine 'i3)            ;Labelled as i3
X
X(SetInputLine 'i3 1)            ;Input i3 is permanently set to 1.
X(ConnectInputToNeuron 'i1 'n1 1)    ;Connect input 1 to neuron 1
X(ConnectInputToNeuron 'i2 'n1 1)    ;Connect input 2 to neuron 1
X(ConnectInputToNeuron 'i3 'n1 -1.5)    ;Connect input 3 to neuron 1
X
X
X;;; Now show the four outcomes of a two input AND function.
X
X(printf "Input: 1 1\n")
X(SetInputLine 'i1 1)        ; Set inputs 1 and 2 to 1
X(SetInputLine 'i2 1)
X(UpdateNetwork 10 '(or1))    ; Update the system.
X(printf "Output: " (NeuronOutput 'n1) "\n")
X
X(printf "Input: -1 1\n")
X(SetInputLine 'i1 -1)        ; Set inputs 1 and 2 to 1,-1
X(SetInputLine 'i2 1)
X(UpdateNetwork 10 '(or1))    ; Update the system.
X(printf "Output: " (NeuronOutput 'n1) "\n")
X
X(printf "Input: -1 -1\n")
X(SetInputLine 'i1 -1)        ; Set inputs 1 and 2 to -1,-1
X(SetInputLine 'i2 -1)
X(UpdateNetwork 10 '(or1))    ; Update the system.
X(printf "Output: " (NeuronOutput 'n1) "\n")
X
X(printf "Input: 1 -1\n")
X(SetInputLine 'i1 1)        ; Set inputs 1 and 2 to 1,-1
X(SetInputLine 'i2 -1)
X(UpdateNetwork 10 '(or1))    ; Update the system.
X(printf "Output: " (NeuronOutput 'n1) "\n")
SHAR_EOF
echo "extracting neural.lsp"
sed 's/^X//' << \SHAR_EOF > neural.lsp
X;;; @@FILE neural.lsp
X;;; Neural Network Construction Set (Vr 1.0 : 5/5/89)
X;;; Copyright (C) 1989 by Jason Leigh  5/5/89
X
X;;; Comments, questions and donations may be sent to:
X;;; Jason Leigh
X;;; 538 Mackenzie Hall, Wayne State University, Detroit, MI 48202.
X;;; email: [e-mail]
X
X;;; Redistribute this program at will.  If you decide to use this program
X;;; as part of some commercial package you are developing, by all means
X;;; proceed, all I ask is that you give me a small amount of credit for it.
X
X;;; This was written as a practice session in using XLISP objects.
X;;; Neurons are ideally suited to object oriented programming since
X;;; each neuron must contain its own current set of attributes.
X;;; See the files: and.lsp, xor.lsp and xor2.lsp for implementations of
X;;; AND and XOR functions using neural networks.  It is the best
X;;; way to learn how to use the PUBLIC functions below.
X
X;;; Fool-proof code is at a minimum so if you don't use these
X;;; functions just the way they are supposed to be used, you'll
X;;; get a lot of error messages by XLISP.
X
X;;; @@SECTION printf
X;;;
X;;; @@REQ
X;;; parameters
X;;;    Any LISP s-expression to be printed.  Allows multiple expressions.
X;;; e.g. (printf "hello there" 1 2 3 "wow \n")
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Works like printf in C or print in BASIC allowing multiple
X;;; expressions to be printed on one line.  Far more convenient than
X;;; prin1, princ etc...
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun printf (&rest parameters)
X    (dolist (item parameters)
X        (princ item)))
X
X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
X
X;;; Network is an a-list of neuron number and neuron object.
X;;; PRIVATE
X(setq *network* ())
X
X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
X;;;             CLASS DEFINITIONS FOR NEURONS
X
X;;; Class of a neuron contains the instance variables:
X;;; threshold, sum and list of input connections.
X;;; threshold is the threshold value associated with each neuron
X;;; to determine whether the neuron should fire or not.  This is
X;;; usually determined empirically.
X;;; sum is the scalar product of all the inputs and connection strengths
X;;; entering a neuron.
X;;; listOfInConnections is an a-list of connections that go into a
X;;; particular neuron and the connection strength associated with the
X;;; connection.
X
X;;; PRIVATE
X(setq NeuronClass (send Class :new
X        '(threshold sum listOfInConnections)))
X
X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
X;;;                METHODS
X
X;;; @@SECTION NeuronClass:output
X;;;
X;;; @@REQ
X;;; None
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; This method
X;;; calls the appropriate function to compute the output of a
X;;; neuron.  Uses the sum and threshold instance variables
X;;; of the given object.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; There are several possible
X;;; output functions that may be used.  These are listed as:
X;;; Relay, Linear and Sigmoid.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :output ()
X    '((if (null listOfInConnections) sum
X                     (OutputFunction sum threshold))))
X
X;;; @@SECTION Relay
X;;;
X;;; @@REQ
X;;; sum
X;;;    scalar product of all inputs and weights.
X;;; threshold
X;;;    threshold of neuron.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Output function of a neuron.  The relay function
X;;; sends a 1 if the sum >= the threshold and -1 otherwise.
X;;; This conforms to the ADALINE.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(defun Relay (sum threshold)
X    (if (>= sum threshold) 1 -1))
X
X;;; @@SECTION Linear
X;;;
X;;; @@REQ
X;;; sum
X;;;    scalar product of all inputs and weights.
X;;; threshold
X;;;    threshold of neuron.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Output function of a neuron.  The linear function
X;;; does not use the threshold but rather sends the sum directly
X;;; as output; I believe this is used in the Neocognitron model.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(defun Linear (sum threshold)
X    sum)
X
X;;; @@SECTION Sigmoid
X;;;
X;;; @@REQ
X;;; sum
X;;;    scalar product of all inputs and weights.
X;;; threshold
X;;;    threshold of neuron.
X;;;
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Output function of a neuron.  The sigmoidal function
X;;; goes toward 0 for large -ve numbers and 1 for large +ve numbers
X;;; This is suitable for the Back-propagation model where a
X;;; semi-linear continuous output function is needed.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(defun Sigmoid (sum threshold)
X    (/ 1.0 (+ 1.0 (exp (- 0.0 (+ (float sum) threshold))))))
X
X;;; @@SECTION NeuronClass:clear
X;;;
X;;; @@REQ
X;;; None.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; This method sets the SUM instance variable of an object to zero.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :clear () '((setq sum 0)))
X
X;;; @@SECTION NeuronClass:setThreshold
X;;;
X;;; @@REQ
X;;; value
X;;;    value to set the threshold.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    This method changes the current threshold value of a
X;;; neuron.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :setThreshold '(value) '((setq threshold value)))
X
X;;; @@SECTION NeuronClass:addConnection
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Unique identifier that specifies an existing neuron.
X;;; weight
X;;;    Connection strength.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Adds the neuron's Id and the connection strength between the
X;;; neuron and the current neuron to the current neuron's list
X;;; of input connections.
X;;; The list of in connections is an a-list of neuron number and
X;;; weight.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :addConnection '(neuronId weight)
X
X    '((setq listOfInConnections
X         (append listOfInConnections
X             `((,neuronId ,weight))))))
X
X;;; @@SECTION NeuronClass:getConnection
X;;;
X;;; @@REQ
X;;; srcNeuronId
X;;;    neuron Id
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Retrieves the neuronId and weight from the
X;;; list of connections associated with a neuron object.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;;
X;;; @@END-STATUS
X
X(send NeuronClass :answer :getConnection '(srcNeuronId)
X    '((assoc srcNeuronId listOfInConnections)))
X
X;;; @@SECTION NeuronClass:removeConnection
X;;;
X;;; @@REQ
X;;; neuronIdToRemove
X;;;    The neuron Id to search for in the list of input connections
X;;; associated with a neuron object.
X;;; destNeuronId
X;;;    The neuron object to remove it from.
X;;; Returns T if connection to remove exists; () otherwise.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Removes a connection between two neurons.
X;;; This simply  requires going to the destination neuron and removing
X;;; the entry in the a-list ListOfInConnections.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :removeConnection '(neuronIdToRemove destNeuronId)
X    '((let* ((connection (send (GetNeuron destNeuronId)
X                  :getConnection neuronIdToRemove))
X            (remaining (Delete connection
X                   listOfInConnections :test 'equal)))
X            (if (null connection)
X           (progn
X             (printf "No connection: " neuronIdToRemove "->"
X                         destNeuronId
X                         " exists.\n")
X             ())
X               (progn
X             (setq listOfInConnections remaining)
X             T)))))
X
X;;; @@SECTION NeuronClass:setSum
X;;;
X;;; @@REQ
X;;; value
X;;;    Value to set the sum.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Set the SUM instance variable of a neuron object to a particular
X;;; value.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :setSum '(value)
X    '((setq sum value)))
X
X;;; @@SECTION NeuronClass:getSum
X;;;
X;;; @@REQ
X;;; None
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Get the SUM instance variable of a neuron object.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :getSum ()
X    '(sum))
X
X;;; @@SECTION NeuronClass:update
X;;;
X;;; @@REQ
X;;; None.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Method to update the sum in a neuron based on connections
X;;; from inputs of other neurons.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X(send NeuronClass :answer :update ()
X    '(
X      (if (not (null listOfInConnections))
X      (progn
X        (setq sum 0)
X        (dolist (connection listOfInConnections)
X        (setq sum (+ sum
X         (* (NeuronOutput (car connection))
X            (cadr connection)))))))))
X
X
X;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
X;;;        HIGH LEVEL INTERFACES TO METHODS
X
X;;; @@SECTION ConnectNeuron
X;;;
X;;; @@REQ
X;;; srcNeuron
X;;; destNeuron
X;;; weight
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Form a connection between the srcNeuron and destNeuron
X;;; that has a connection strength given by weight.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; Connect a source neuron to a destination neuron using a weight.
X;;; PUBLIC
X(defun ConnectNeuron (srcNeuron destNeuron weight)
X    (send (GetNeuron destNeuron) :addConnection srcNeuron weight))
X
X;;; @@SECTION CreateNeuron
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Unique Id to give to neuron.
X;;; threshold
X;;;    Threshold to set in the neuron.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Create a new neuron that is identified by neuronId.
X;;; If you have a friend who you think has the intelligence of a
X;;; single neuron, use his/her name!
X;;;
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun CreateNeuron (neuronId threshold)
X    (if (GetNeuron neuronId)
X       (printf "Identifier: " neuronId " already exists.\n")
X       (progn
X        (setq *network*
X            (append *network* `((,neuronId ,(MakeNeuron)))))
X        (send (GetNeuron neuronId) :setThreshold threshold)
X        (send (GetNeuron neuronId) :clear)
X        neuronId)))
X
X;;; @@SECTION MakeNeuron
X;;;
X;;; @@REQ
X;;; None
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Invoke the :new method to create a new neuron object of class:
X;;; NeuronClass.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; Make a neuron by creating an object from the neuron class.
X;;; PRIVATE
X(defun MakeNeuron ()
X    (send NeuronClass :new))
X
X;;; @@SECTION GetNeuron
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Id of neuron
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Get a neuron from the network.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;;    Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun GetNeuron (neuronId)
X    (cadr (assoc neuronId *network*)))
X
X;;; @@SECTION Examine
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Neuron Id
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Examine the inner parameters of a neuron.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; Examine the inner instance variables of a neuron.
X;;; PUBLIC
X(defun Examine (neuronId)
X    (send (GetNeuron neuronId) :show)
X    (printf "Output: " (NeuronOutput neuronId) "\n"))
X
X;;; @@SECTION CreateInputLine
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Give a unique name to inputs into the network.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Actually an input line is made a neuron with no threshold
X;;; so that whatever goes into the neuron comes straight back out.
X;;; This saves having to declare an input class.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun CreateInputLine (neuronId)
X    (CreateNeuron neuronId 0))
X
X;;; @@SECTION SetInputLine
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Input line Id.
X;;; inputValue
X;;;    Value to set input value to.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Set the output sum of a neuron to always equal some value.
X;;; This disguises the fact that an input line is really a neuron
X;;; with only a set output.
X;;;
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun SetInputLine (neuronId inputValue)
X    (send (GetNeuron neuronId) :setSum inputValue))
X
X;;; @@SECTION ConnectInputToNeuron
X;;;
X;;; @@REQ
X;;; inputNum
X;;;    Input line Id
X;;; destNeuron
X;;;    Destination neuron Id
X;;; weight
X;;;    Strength of connection
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Connect an input line to a neuron.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun ConnectInputToNeuron (inputNum destNeuron weight)
X    (ConnectNeuron inputNum destNeuron weight))
X
X;;; @@SECTION UpdateNeuron
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    neuron id.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Update a neuron by sending update message to neuron object.
X;;; This recalculates the SUM.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PRIVATE
X(defun UpdateNeuron (neuronId)
X    (send (GetNeuron neuronId) :update))
X
X;;; @@SECTION UpdateNetwork
X;;;
X;;; @@REQ
X;;; updateLimit
X;;;    After how many updates through the network should
X;;; results be reported.
X;;; neuronList
X;;;    List  of neuron Id's to examine when updateLimit is reached.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Update the system by updating every neuron.  This occurs
X;;; repeatedly until the network outputs have stablized.
X;;; Since some networks may take a long time to produce results,
X;;; setting an updateLimit and neuronList will allow intermediate
X;;; reporting of results while the network is updating.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun UpdateNetwork (updateLimit neuronList)
X    (let ((change ())
X          (old    ())
X          (updates 0))
X        (loop
X         (setq change ())
X         (dolist (eachNeuronInNetwork *network*)
X        (setq old (NeuronOutput (car eachNeuronInNetwork)))
X        (UpdateNeuron (car eachNeuronInNetwork))
X        (if (not (equal old
X               (NeuronOutput (car eachNeuronInNetwork))))
X               (setq change T)))
X         (setq updates (+ updates 1))
X         (if (> updates updateLimit)
X         (progn
X            (setq updates 0)
X             (ReportNeurons neuronList)))
X         (if (null change) (return)))))
X
X;;; @@SECTION ReportNeurons
X;;;
X;;; @@REQ
X;;; neuronList
X;;;    List of neurons to examine.
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Examine a list of neurons.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun ReportNeurons (neuronList)
X    (cond ((null neuronList) T)
X          (T (progn
X           (printf "\n*****  Neuron Id: " (car neuronList) "\n")
X           (Examine (car neuronList))
X           (printf "\n")
X           (ReportNeurons (cdr neuronList))))))
X
X;;; @@SECTION NeuronOutput
X;;;
X;;; @@REQ
X;;; neuronId
X;;;    Neuron Id
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;;    Return the output of a neuron.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun NeuronOutput (neuronId)
X    (send (GetNeuron neuronId) :output))
X
X;;; @@SECTION ClearNetwork
X;;;
X;;; @@REQ
X;;; None
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Simply sets *network* to null thus losing all neurons defined so far.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; Clear network of all neurons
X;;; PUBLIC
X(defun ClearNetwork ()
X    (setq *network* ()))
X
X;;; @@SECTION ChangeWeight
X;;;
X;;; @@REQ
X;;; srcNeuron
X;;; destNeuron
X;;; newWeight
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Change the connection strength from srcNeuron to destNeuron to the
X;;; new value given by newWeight.  This function was written so that
X;;; feedback networks that learn by modifying their weights based
X;;; on a comparison of target and actual results, can be designed.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun ChangeWeight (srcNeuron destNeuron newWeight)
X    (if (send (GetNeuron destNeuron)
X          :removeConnection srcNeuron destNeuron)
X        (ConnectNeuron srcNeuron destNeuron newWeight)))
X
X
X;;; @@SECTION GetWeight
X;;;
X;;; @@REQ
X;;; srcNeuron
X;;; destNeuron
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Find the connection strength between srcNeuron and destNeuron.
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X(defun GetWeight (srcNeuron destNeuron)
X    (let ((result (send
X               (GetNeuron destNeuron)
X               :getConnection srcNeuron)))
X         (cadr result)))
X
X
X;;; @@SECTION GetSum
X;;;
X;;; @@REQ
X;;; neuronId
X;;; @@END-REQ
X;;;
X;;; @@FUNC
X;;; Find the SUM of a neuron. (i.e. Net)
X;;; @@END-FUNC
X;;;
X;;; @@STATUS
X;;; Complete.
X;;; @@END-STATUS
X
X;;; PUBLIC
X
X(defun GetSum (neuronId)
X    (send (GetNeuron neuronId) :getSum))
SHAR_EOF
echo "extracting neural.readme"
sed 's/^X//' << \SHAR_EOF > neural.readme
X    Amiga Neural Network Construction Set (Vr 1.0 : 5/5/59)
X
XThe following is a brief description of the files contained in
Xa tiny XLISP Neural Network Construction Set (ANNCS):
X
XThe ANNCS is a set of functions that allows quick and easy construction
Xof neural networks.  The system is very simple and as a result much
Xcan be enhanced.  Eventually, if I have nothing better to do over the
Xsummer, I will attach a graphical interface that allows you to
Xvisually create neural networks and convert the images to ANNCS Lisp
Xfunction calls.  This is by no means a competitor for the Rochester
XConnectionist Simulator but hay, what the heck, it's at least running
Xon my favorite computer!
X
XNeural.lsp    : Lisp source code for the construction set.
XAnd.lsp        : Implementation of the AND function using neural networks
X          constructed from functions in Neural.lsp
X          Uses Widrow's ADALINE's
XXor.lsp        : Implementation of the XOR function using neural networks
X          constructed from functions in Neural.lsp
X          Uses Widrow's ADALINE's
XXor2.lsp    : Alternate implementation of the XOR function using
X          neural networks constructed from functions in Neural.lsp
X          Uses Rumelhart's network.
X
XSorry there is no demonstration of learning in the Backpropagation
Xmodel but it can easily be constructed using the functions in this
Xconstruction set; so why don't you do it and send me a copy!
X
XTo load the programs, fire up XLISP and type:
X
X(load "Neural.lsp")
X(load "And.lsp")
X(load "Xor.lsp")
X(load "Xor2.lsp")
X
X*************************************************************************
XFunctions that are present in version 1.0 of ANNCS:
X
XRead And.lsp, Xor.lsp and Xor2.lsp for a demonstration of how the
Xfunctions may be used.
X
X(ClearNetwork)
X    - Clear the network of all neurons.
X
X(CreateNeuron neuronId threshold)
X    - Create a neuron and give it a name: neuronId
X      Set the threshold value to: threshold.
X      e.g. (CreateNeuron 'myNeuron 2.3)
X
X(CreateInputLine inputId)
X    - Create an input line and give it a name: inputId.
X      e.g. (CreateInputLine 'input1)
X
X(SetInputLine inputId inputValue)
X    - Set the input value to the input line: inputId to the
X      value: inputValue.
X      e.g. (SetInputLine 'input1 1)
X
X(ConnectInputToNeuron inputId destNeuron weight)
X    - Connect an input line to a neuron.  The weight is the
X      connection strength between the input line and neuron.
X
X(ConnectNeuron srcNeuron destNeuron weight)
X    - Connect two neurons together.  I.e. Output of
X      srcNeuron feeds into input of destNeuron with a connection
X      strength of: weight.
X
X(GetNeuron neuronId)
X    - Get a neuron with the given Id name.  This can be used to
X      get an input line as well.
X      e.g. (GetNeuron 'myNeuron)
X      Returns the neuron object.
X
X(Examine neuronId)
X    - Examine the contents of a neuron or input line.
X      e.g. (Examine 'input1)
X
X(ReportNeurons neuronList)
X    - Like examine except a list of neuron Id's are given.
X      e.g. (ReportNeurons '(input1 'or1))
X
X(UpdateNetwork updateLimit neuronList)
X    - Based on the current inputs and connections, update the
X      entire network so that we can see what the output
X      should be.  In some networks it may take a considerable
X      number of iterations before the network finally stablizes
X      to a result.  The parameter updateLimit is an integer
X      that determines how many iterations the system is to
X      update before the list of neuron Id's in neuronList are
X      examined.  Updating then resumes.
X
X
X(NeuronOutput neuronId)
X    - Return the output of a particular neuron.
X
X(ChangeWeight srcNeuronId destNeuronId newWeight)
X    - Change the connection strength between two neurons
X      or an input and a neuron.  I put this in so that if you
X      wanted to construct a backpropagation network you at least
X      had the means to train it.
X
X(GetWeight srcNeuronId destNeuronId)
X    - Get the connection strength between srcNeuronId and destNeuronId.
X
X(GetSum neuronId)
X    - Get the internal SUM (or NET) of a particular neuron.
X
X*************************************************************************
XRedistribute this program at will.  If you decide to use this program
Xas part of some commercial package you are developing, by all means
Xproceed, all I ask is that you give me a small amount of credit for it.
X
XSend comments, ideas, questions, improvements. donation$ to:
XJason Leigh
X538 Mackenzie Hall, Wayne State University, Detroit, MI 48202
Xe-mail: [e-mail]
SHAR_EOF
echo "extracting xor.lsp"
sed 's/^X//' << \SHAR_EOF > xor.lsp
X;;; @@FILE and.lsp
X;;; Copyright (C) 1989 By Jason Leigh 5/5/89
X
X;;; Redistribute this program at will.  If you decide to use this program
X;;; as part of some commercial package you are developing, by all means
X;;; proceed, all I ask is that you give me a small amount of credit for it.
X
X;;; XOR created with 2 layer MADALINES.  This uses the most basic
X;;; AND and OR ADALINES and connects them up into the form of
X;;; the XOR expressions which is: ((not a) and (b)) or ((a) and (not b))
X
X(ClearNetwork)
X
X;;; Select the Relay function as the output function.
X(defun OutputFunction (sum threshold)
X    (Relay sum threshold))
X
X(CreateNeuron 'or1 0)        ; Create a neuron for OR
X(CreateInputLine 'i1)        ; Create input line i1
X(CreateInputLine 'i2)        ; Create input line i2
X(CreateInputLine 'unity)    ; Create input line `unity' which is always 1
X(SetInputLine 'unity 1)
X(CreateNeuron 'and1 0)        ; Create two and neurons.
X(CreateNeuron 'and2 0)
X
X(ConnectInputToNeuron 'i1 'and1 1)     ; Make appropriate connects.
X(ConnectInputToNeuron 'i2 'and1 -1)     ; The best thing to do is to
X(ConnectInputToNeuron 'unity 'and1 -1.5) ; get a piece of paper out
X(ConnectInputToNeuron 'i1 'and2 -1)     ; and draw out the connections
X(ConnectInputToNeuron 'i2 'and2 1)     ; to see what the network
X(ConnectInputToNeuron 'unity 'and2 -1.5) ; actually looks like.
X
X(ConnectNeuron 'and1 'or1 1)
X(ConnectNeuron 'and2 'or1 1)
X(ConnectNeuron 'unity 'or1 1.5)
X
X
X;;; Let's try out the network with the 4 possible binary combinations.
X
X(printf "Input: -1 -1\n")
X(SetInputLine 'i1 -1)
X(SetInputLine 'i2 -1)
X(UpdateNetwork 10 '(or1))
X(printf "Output: " (NeuronOutput 'or1) "\n")
X
X(printf "Input: 1 -1\n")
X(SetInputLine 'i1 1)
X(SetInputLine 'i2 -1)
X(UpdateNetwork 10 '(or1))
X(printf "Output: " (NeuronOutput 'or1) "\n")
X
X(printf "Input: -1 1\n")
X(SetInputLine 'i1 -1)
X(SetInputLine 'i2 1)
X(UpdateNetwork 10 '(or1))
X(printf "Output: " (NeuronOutput 'or1) "\n")
X
X(printf "Input: 1 1\n")
X(SetInputLine 'i1 1)
X(SetInputLine 'i2 1)
X(UpdateNetwork 10 '(or1))
X(printf "Output: " (NeuronOutput 'or1) "\n")
SHAR_EOF
echo "extracting xor2.lsp"
sed 's/^X//' << \SHAR_EOF > xor2.lsp
X;;; @@FILE xor2.lsp
X;;; Copyright (C) 1989 By Jason Leigh 5/5/89
X
X;;; Redistribute this program at will.  If you decide to use this program
X;;; as part of some commercial package you are developing, by all means
X;;; proceed, all I ask is that you give me a small amount of credit for it.
X
X;;; Creating an XOR network using the functions in neural.lsp
X
X;;; The activation function is Sigmoid and a diagram of this
X;;; network can be found in:
X;;; Parallel Distributed Processing Volume I: Foundations
X;;; By David E. Rumelhart, James L. McClelland
X;;; p.331.
X
X(ClearNetwork)
X
X;;; Select the Sigmoid function as the output function.
X(defun OutputFunction (sum threshold)
X    (Sigmoid sum threshold))
X
X(CreateNeuron 'out 6.3)
X(CreateNeuron 'mid 2.2)
X(CreateInputLine 'in1)
X(CreateInputLine 'in2)
X(ConnectInputToNeuron 'in1 'mid -6.4)
X(ConnectInputToNeuron 'in1 'out -4.2)
X(ConnectInputToNeuron 'in2 'mid -6.4)
X(ConnectInputToNeuron 'in2 'out -4.2)
X(ConnectNeuron 'mid 'out -9.4)
X
X(printf "Input: 0 0\n")
X(SetInputLine 'in1 0)
X(SetInputLine 'in2 0)
X(UpdateNetwork 10 '(mid out))
X(printf "Output: " (NeuronOutput 'out) "\n")
X
X(printf "Input: 1 0\n")
X(SetInputLine 'in1 1)
X(SetInputLine 'in2 0)
X(UpdateNetwork 10 '(mid out))
X(printf "Output: " (NeuronOutput 'out) "\n")
X
X(printf "Input: 0 1\n")
X(SetInputLine 'in1 0)
X(SetInputLine 'in2 1)
X(UpdateNetwork 10 '(mid out))
X(printf "Output: " (NeuronOutput 'out) "\n")
X
X(printf "Input: 1 1\n")
X(SetInputLine 'in1 1)
X(SetInputLine 'in2 1)
X(UpdateNetwork 10 '(mid out))
X(printf "Output: " (NeuronOutput 'out) "\n")
SHAR_EOF
echo "End of archive 1 (of 1)"
# if you want to concatenate archives, remove anything after this line
exit